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ABSTRACT 

This report describes Copenhagen (Denmark) Business 
School Library's (CBSL) use of performance measurement to manage the 
quality of library services and facilities. The CBSL quality 
management program consists of: a statement of strategic goals of 
development and operation, including a focus on users, good service, 
efficient internal functions, and optimal use of resources; a program 
for staff training and development with plans for manuals for 
operations and service; an evaluation procedure; and a procedure for 
corrective action. Performance measurement is a necessary instrument 
for quality management. It makes possible the assessment of whether 
or not a library fulfills its goals and objectives, and whether or 
not particular objectives remain relevant. Performance measurement 
cannot provide a totally objective and comprehensive image of a 
library's function, but is still necessary for a quality conscious 
library. Management is an art, not a science, and decisions must be 
made on the basis of approximate knowledge and intuition. The report 
covers: (1) librarians and management theories; (2) performance 
measurement, evaluation, strategy, corrective action, and the CBSL 
quality management program; (3) the concept, purpose, and types of 
performance measurement; (4) a fictional case study measuring 
performance of an interlibrary loan program; and (5) problems of 
performance measurement that affect quality management. A brief 
discussion with the author is provided. (Author/SWC) 
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1 . Introduction: Librarians and 
Management Theories 

1.1 THE PROLIFERATION OF MANAGEMENT 
SYSTEMS 

As you are aware, not a month goes by without the 
appearance of a new management theory, a new 
management guru, a new management bestseller. 
Sometimes, the newness consists in demolishing 
well-established theories and practices of other 
gums, like Heiuy Mintzberg’s The rise and fall of 
strategic planning, which arrived at the Copenhagen 
Business School (CBS) just after its governing body 
had decided to develop a strategic plan. As 
Mintzberg is one of the management gums held in 
esteem at the CBS, his book was received with 
somewhat mixed feelings. 

In later years, I have myself encountered manage- 
ment by objectives, participatory management, self- 
governing groups, team-building, consensus man- 
agement (which I absolutely loathe and consider to 
be a violation of the human right of disagreement 
and of the noble culmre of contradiction), strategic 
plaiming, corporate mm-around, the virtual corpora- 
tion, the slim corporation, management by walking 
around, benchmarking, mentoring, the five-minute 
manager, total quality management and ISO quality 
certification. Most recently I have read about 180 
degrees management and - a month later - 360 
degrees management. What these last can be about, 

I do not know, but I am quite sure that at the next 
■ IFLA conference somebody is going to stand up and 
give a paper on ‘Management by degrees in acade- 
mic libraries’. 

We should not wonder at this seeming fecundity 
of management theory. It is, after all, not only a 
field for fertile brains, but also a major industry, 
involving big money. 

Like the fashion industry, the management indus- 
try systematically depends on continuous change 
and newness. The gurus are in a perpetual state of 
competition for lucrative consultancies, grand 
speaking fees, and next month’s books on the best- 
seller list. 

Actually, competition can sometimes turn into the 
ludicrous or the ethically questionable. For exam- 



ple, a recent book entitled The discipline of the mar- 
ket leaders stayed for 15 weeks on the New York 
Times bestseller, reaching as high as number five. It 
was then discovered that the authors’ consultancy 
firm had by various means bought between 40,000 
and 50,000 copies of the book from bookshops all 
over the US, particularly bookshops thought to be 
monitored by the New York Times (cf. The 
Economist, August 5th 1995, p. 53). So, beware 
of bestsellers. 

1 .2 LIBRARY MANAGERS' FASCINATION WITH 
MANAGEMENT SYSTEMS 

Instead of wondering at the proliferation of manage- 
ment theories one might more reasonably wonder at 
library managers’ fascination with them, particular- 
ly in the English-speaking world. 

Whenever a new system is presented, it does not 
take many months before articles begin to appear in 
library periodicals on their application in libraries. 

There is a positive side to this fascination: it rep- 
resents an honest recognition that many, maybe 
most of us, are not bom managers, and that manage- 
ment skills can actually to a considerable extent be 
learnt or developed. Also, the use of one system as a 
frame of reference can help to develop one’s own 
consciousness about management issues in general 
and in particular. 

On the negative side, the fascination with man- 
agement theory is probably also due to feelings of 
insecurity and inadequacy, to some extent but not 
exclusively caused by the major technological 
upheavals in the library world. 

Such fascination may lead to blind acceptance of 
one system, uncritical dependence on the guru, cog- 
nitive blindness towards divergent stimuli from the 
environment and self-imposed limitation on avail- 
able options for action. 

1 .3 'BHIKKUS, YOU MUST BE YOUR OWN 
SALVATION' 

'There is no saviour system out there, waiting to be 
discovered and able to solve one’s management 
problem. Learn from the hinayana Buddha’s mes- 
sage to his monks: Bhikkus, you must be your own 
salvation! 

In conclusion, a library manager should carefully 
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and critically select all relevant management 
insights, apply them to her own library and create a 
flexible management system adapted to local and 
personal circumstances. 

This is what we have been trying to do at the 
Copenhagen Business School Library (CBSL). 

2. From Problem Solving to Quality 
Management 

T he development of performance measurement 
and quality management at the CBSL wiU be 
related here. Through this process various important 
elements of quality management have been identi- 
fied and related to each other in a way which may 
be of interest to other libraries pondering whether 
and how to proceed in this area. 

At the outset, it has to be admitted that this devel- 
opment as a whole was not the result of a major 
planning exercise based on an overall view of things 
and a deep knowledge of management theory. The 
perspective, though, was clear: how things look to 
the user. It turned out that one phase of development 
naturally led to another, and in the end we have 
come up with a fairly coherent ‘system’, even 
though we somehow did it all backwards. 

2.1 V/HAT ARE WE DOING? PERFORMANCE 
MEASUREMENT 

In 1988/89 the CBSL was confronted with a number 
of serious problems: other libraries in the area were 
constantly complaining about being swamped by 
CBS-students (about 15,0(X)) seeking service they 
seemingly could not get in their own library; the 
budget was grossly inadequate in terms of the tasks 
demanded of the library; the major transition to 
computerisation was becoming urgent; there was a 
feeling that internal processes were not efficient. 

In this situation it was decided that the flrst prior- 
ity would be to get to know what we actually were 
doing. 

We already had a set of basic statistics, ie. those 
reported on a yearly basis to the Danish library sta- 
tistics published by the National Librarian’s Office. 
These statistics comprised essential figures on col- 
lections, loans and resources. 

To these statistics were added a series of data on 
output, eg. fin rate and delivery times for inter- 
library loans from other libraries, and collection use. 
Special emphasis was laid on data concerning the 
availability of books. 

To the output data were added data on internal 
processing functions, a major one being the time it 
takes for new books to reach the shelves after being 



received in the acquisitions department. This rather 
simple measure actually reflects the overall fimc- 
tioning of a number of operations, acquisition, clas- 
sification and indexing, descriptive cataloguing, 
physical preparation and registration in the lending 
module of the computer system. 

The development of the performance measure- 
ment system is an ongoing process. It received new 
impetus at later stages with the addition of data on 
efficiency and effectiveness of operations and ser- 
vice as well as on the composition of the user group. 
Every year some measures are cancelled, new mea- 
sures established and others revised even though the 
modifications are now comparatively few from year 
to year. 

Next year we shall be looking at the performance 
measures proposed by ISO and IFLA and take over 
what seems useful for us. Also a serious attempt at 
including cost data has to be made. 

Our performance measurement system probably 
never will become completely finished since new 
developments in service and working procedures 
will have to be monitored, but a stable form is 
expected to be established in two to three years. 

2.2 ARE WE DOING IT WELL? EVALUATION 

Quite soon in the work with performance measures, 
it became apparent that they did give us a much bet- 
ter picture of what we were doing as a library but 
that used properly they were limited to the descrip- 
tive sphere. They did not in themselves teU us if we 
were doing a good job. 

However, to management it was important to be 
able to pass a value judgment on the work of the 
library, to be able to conclude whether or not we 
were doing a good job, and if this or that particular 
activity was satisfactory or not. 

In connection with a set of fortuitous external cir- 
cumstances, among them the interest and support in 
a project on the part of the Ministry of Education, it 
was decided to establish a program of library evalu- 
ation. The main purpose of the project was to evalu- 
ate the ‘goodness’ of the library. The program was 
to be ongoing and aim at armual or biaimual evalua- 
tions. 

To ensure independent expertise, Maurice Line 
was engaged as a consultant, which has led to a very 
fruitful and constructive relationship. 

The first evaluation exercise was performed in 
1991/92. It was based on a set of performance mea- 
sures and - to a limited extent - on qualitative data 
fiom interviews with users. The third evaluation 
exercise covering 1994/95 has just ended. As with 
our performance measurement, the evaluation pro- 



Michael Cotta-Schenberg: PM in the Context of Quality Management 



cedure is developed continually but will probably 
find a more stable form in two to three years. 

2.3 ARE WE DOING WHAT WE WANT TO DO? 

STRATEGY 

Already in connection with the first evaluation exer- 
cise it became apparent that subjective evaluation 
based on the managers’ more or less conscious or 
articulated perceptions and values are satisfactory 
neither in general nor to the managers themselves. 
For a value judgment to work in an organisational 
context of our type, clear criteria of value should be 
formulated and approved by the responsible body, in 
our case the Library Board. 

Three types of such criteria can be distinguished: 
Firstly, there are norms and standards approved by 
competent external authorities and associations. 
Examples are charters for public library services 
issued by local authorities in England, and guide- 
lines and norms for library services finm American 
library associations. Secondly, best practice may be 
used, obtained by comparison with other libraries 
(benchmarking) or by comparison of one’s own 
performances at different times, mostly from year to 
year. Thirdly, many libraries now have a set of 
clearly formulated goals in the form of strategies, 
plans or sometimes service level agreements, often 
negotiated with and approved by the library’s acade^ 
mic mother institution. 

At the CBSL, we chose the last option in the form 
of a strategic plan. We had earlier had to fomiulate 
some goals in connection with long-range plarming 
initiatives for the whole Business School. These 
were developed into a fully-fledged plan covering 
the whole spectrum of library activities. The plan is 
revised annually and approved by the Library Board 
as well as by the Academic Council, the highest 
governing board of the Business School. 

Most often strategy is considered to be an expres- 
sion of major and long-range lines of change and 
development. We feel, however, that for a service 
institution of our type daily operations and service 
are a major strategic issue. The plan therefore has 
two dimensions. One dimension is development 
both in terms of the long-range transformation of 
the library from a book-place to an information 
nexus and of short-range concrete initiatives of 
development and improvement. Another dimension 
is operation , which covers not only service - more 
or less embodying a service level agreement - but 
also the daily internal functions which support the 
service function. 

In its present form the Strategic goals of develop- 
ment and operation of the CBSL comprise a short 



Mission statement and four major Goals, being 
Focus on users. Good service. Efficient internal 
functions and Optimal use of resources. The goals 
are made concrete in 60 objectives, each with sub- 
objectives and specific criteria of success when rele- 
vant. 

The strategic plan is now approaching a stable 
form, though the concrete projects of development 
as well as the specific success criteria will change. 

After the development of the strategic plan the 
yearly evaluation report has been based on this plan. 
It is structured after the four major Goals and is 
divided into 60 sections, one for each objective. 
Each section contains 

- a statement of the objective with sub-objectives 
and success criteria if any 

- performance measures 

- results of any special studies of the activity in 
question during the year 

- a summary of initiatives of development, 
improvement or correction concerning the activ- 
ity in question during the year 

- an evaluative statement concerning both the con- 
formance between the approved objective and 
the actual practice and any problems indicated 
by the performance measures or special studies 

At this stage of development, ie. in 1994, the 
Business School decided to develop an overall qual- 
ity program for the whole school, with the assis- 
tance of 

a British specialist, Lee Harvey. 

We had, therefore, to look into the quality man- 
agement literature and movement. We then decided 
that with our strategic plan, performance measures 
and evaluation procedure we already had some of 
the basic elements of a quality management pro- 
gram. However, two elements of such a program 
might be considered to be missing or insufficiently 
developed, one being the creation of conditions for 
quality and the other being ‘corrective’ action. 

As for creating the conditions, we already had a 
formalised procedure for ensuring continuing staff 
training and development and some manuals for 
internal operations, but these will have to be devel- 
oped further as an integral part of overall systematic 
quality management 

2.4 WHAT IF WE DON'T DO WHAT WE WANT 
TO DO? CORRECTIVE ACTION 

As for corrective action, we had of course earlier 
done something about functions which were evalu- 
ated negatively, but this procedure, too, did not have 
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the degree of consistency and formalisation required 
by a quality management system. 

In connection with our evaluation report for 1995 
we have begun to develop this element. We shall 
probably have to find another name for it, since 
‘corrective action’ - at least in Danish - has connota- 
tions of something unpleasant done in penal institu- 
tions! 

In case of a negative evaluation of some activity, 
three types of response are possible. 

Firstly, due to problems with measuring perfor- 
mance - to be described in part 5 of this paper - it is 
necessary to consider whether the performance mea- 
sures are correct or sufficient. If not, it may be 
decided to modify the performance measurement in 
order to obtain better or more data. 

An example: performance measures showed that 
we did not fulfil our success criteria of a minimum 
95% fiU rate of requisitions for interlibrary loans 
from other libraries. The responsible department 
was convinced that the fill rate was better than indi- 
cated by the performance measures and a closer 
look at the procedure for collecting the data actually 
showed that there were some problems both of 
validity and reliability which resulted in inexact fig- 
ures giving a more negative picture than the actual 
case. As it is usually the other way round, we dou- 
ble-checked the findings and finally concluded that 
we had to modify the procedure and get better per- 
formance measures before being able to make an 
evaluation. 

Secondly, if the performance measures stand up 
under inspection you should consider whether the 
negative evaluation may be due to impossible or 
obsolete goals, objectives or success criteria. In that 
case you should change the goal instead of the prac- 
tice . An example: we had a success criterion con- 
sisting in a maximum mean delivery period of 
books from booksellers of one month. For three suc- 
cessive years, this criterion was not fulfilled. Our 
only option however seemed to be to shorten the 
time by having books from the US delivered by 
airmail and generally using rush orders much more 
extensively than now. However, this option is very 
costly so we have decided to retain our present 
practice and to lengthen the delivery time specified 
by the success criterion. Such a course may be dan- 
gerous, of course; stiU it is sometimes necessary: 
goals and objective are not sacrosanct, they too 
should be under continuous scrutiny and revision. 

Thirdly, if one is satisfied with the quality of 
one’s performance measures and wishes to uphold 
the existing goals, objectives and success criteria 
one will obviously have to initiate remedial mea- 



sures . If this is not done, the whole quality process 
loses its meaning and credibility. 

A remedial measure may take the form of rein- 
forcing the existing procedures . I believe it is the 
second law of thermodynamics which can be restat- 
ed to mean that things fall apart naturally and by 
themselves. There is always a tendency not to 
uphold existing rules, not necessarily because staff 
have decided that they are not good. Rules which 
are not actively upheld and reinforced will eventual- 
ly fall into disuse. 

On the other hand, if a procedure is not observed 
it may be because staff at grassroots level have just- 
ly determined that something is wrong with the 
procedure. In that case one should abolish, change 
or develop the procedure - in consultation with rele- 
vant staff, evidently. 

If the problem is due to staff lacking the neces- 
sary qualifications or motivation for implementing 
sensible procedures, staff development and training 
measures are indicated, and if that does not help it 
may be necessary to shift staff to other tasks. 

2.5 THE QUALITY AMNAGEMENT PROGRAM OF 
THE CBSL- 1995 

As you have heard, we have not only done it aU our 
way, but we also did it backwards, starting from a 
problem and not from a decision to have a quality 
management system. 

Our quality program is stiU in a period of devel- 
opment. It may attain a more stable form in some 
years, but it wUl most probably be in a state of con- 
tinuous change rwt only caused by new insights 
from management theory, but also by changes in the 
library and information service environment. 

As of this year, our quality program contains four 
major elements: 

- a statement of strategic goals of development and 
operation consisting of a mission statement, 4 
major goals, 60 concrete objectives with sub- 
objectives and specific criteria of success when 
relevant 

The objectives under the major goal of Good 
service together form the substance of a service 
level agreement with the Business School 

- a program for staff training and development and 
for developing a set of manualsjfor operations 
and service 

- an evaluation procedure based on performance 
measurement and other relevant information 

- a procedure for corrective action comprising 
revision of performance measures, revision of 
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goals, objectives and success criteria, and reme- 
dial measures in the form of reinforcing or 
developing procedures and specific staff training 
and development 

Over the last year, we have pondered whether we 
should go further and implement a Total quality 
management program or seek quality certification 
based on ISO 9000. Right now it is my feeling that 
we should not do this. For two reasons: firstly, I 
believe that our quality program as it stands today 
gives us the essentials of quality management and 
that we would not benefit truly by submitting to the 
formal requirements of TQM and ISO certification. 

Secondly, we really cannot expand further the 
resources spent on this whole process. Some we 
should use, definitely, but we must also ensure ade- 
quate resources for current operations and the - 
very - important development tasks necessary dur- 
ing these years. 

Thirdly, there is the matter of perspective. As 
managers we do not want ourselves or our staff to 
become engrossed in procedures of operations and 
measurements, evaluation and in general control. 
Even more important is it to keep the institution 
tuned to the winds of change and actively engaged 
in the major upheavals affecting the library and 
information world. These issues are of course close- 
ly related to quality and quality management, but 
they are of a different order. Concern with quality 
and control must support the even more vital issues 
and not deflect our energy from them. 

3. Concept, Purpose and Types of 
Performance Measurement 

T he following remarks are not intended to be of a 
theoretical nature, but to reflect the experience 
of a practitioner and his attempts to develop a useful 
concept of PM. 

3.1 THE CONCEPT OF PERFORMANCE 
MEASUREMENT 

3.1.1 Terminological flux 

As you are all aware, PM is still in a considerable 
terminological flux. The important point is that 
there is not any ‘correct’ terminology. 

Quite probably, the national and international 
efforts of developing standards and toolkits for PM 
in libraries (The ALA guidelines in the US, 
SCONUL guidelines and The Effective Academic 
Library in the UK, ISO and IFLA internationally) 
will eventually lead to some harmonisation of ter- 
minology. 



Total harmony will never prevail, though. Certain 
terms are untranslatable from or into English, the 
international language of librarianship. For exam- 
ple, the very term ‘performance measurement’ 
translates directly into a Danish term ‘praestations- 
mMing’ which has somewhat negative connotations 
in our culture. Also the distinction between ‘effi- 
ciency’ and ‘effectiveness’ carmotbe rendered 
directly into other European languages which have 
only retained one substantive based on the Latin 
verb efficere and its related nouns. And the distinc- 
tion between ‘measures’ and ‘indicators’ has given 
rise to a definitional controversy in Great Britain 
Miich I do not consider to be important. It does 
have a certain entertainment value, though, and it is 
one of those extravagancies of the British intellect 
which may assure their neighbours that the British 
are not quite as pragmatic as they would have us 
believe [murmurs and groans from the audience!]. 

3.1.2 What it is not 

In the literature, performance measurement and 
evaluation are sometimes used interchangeably. 

This is one practice I consider to be misleading, at 
the risk of contradicting my own dictum that there is 
no ‘correct’ use. The reason is that I believe it is 
essential to try as much as possible to keep descrip- 
tion, of which measurement forms part, distinct 
fi'om value judgment, which is evaluation. 

Another thing that it is not is problem solving . Of 
course, knowledge about the problem and its extent, 
provided through performance measurement, is 
essential but the solution to the problem is another 
thing completely and it is chosen in consideration of 
policy factors and other factors, sometimes quite 
unrelated to the problem as such. The performance 
measures may indicate relevant elements of problem 
solving; they do not provide the solution. 

3.1.3 WhatiLis 

By performance I understand library activities in the 
broad sense. 

Measurement is generally understood to involve 
some form of counting. The problem is: what may 
be counted? Evidently entities of a physical nature, 
eg. things like books, persons or users may be 
counted, and transactions too. 

What about users’ attitudes, opinions or satisfac- 
tion? And staff’s for that matter or other stakehold- 
ers’, too? These phenomena are of a psychological 
nature, and they carmot be measured directly. 
Quantified expressions of the phenomena may 
be obtained, but what is counted is how many peo- 
ple tell us this or that about their sentiments and 
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experiences and not the sentiments and experiences 
themselves. 

Also, the matter of the degree to which users 
have certain sentiments (eg. very satisfied, satisfied, 
dissatisfied, very dissatisfied) cannot be measured 
directly, only how many people use the different 
answer categories. And of course the categories in 
such a division of sentiments (very satisfied through 
very dissatisfied) are not equidistant at aU. 

Some research on users’ perceptions of service 
does not aim at quantifiable data at all, but at exem- 
plary qualitative description and analysis. Indeed, 
some would altogether give up the hopeless task of 
quantifying anything - as if qualitative research did 
not, also, suffer from unsolvable methodological 
problems in terms of knowledge and generalisation. 

This whole matter is naturally an interesting 
problem for specialists on methods in the social ser- 
vices. 

On the level of the day-to-day operation of 
libraries I think it is reasonable to include data on 
users’ satisfaction etc. xmder the concept of mea- 
sures as long as it is clearly understood and remem- 
bered that the measures are not direct measures but 
only indirect expressions of psychological states. 
Also that they are based on question and answer cat- 
egories which users understand and relate to differ- 
ently. The same apphes to categorisation of degree 
(eg. much, httle, often, seldom). Personally, I find 
that these problems are fundamentally unsolvable, 
and that you have to live with them if you want to 
include data on how people experience our service 
functions and institutions. 

In conclusion and for the purpose of this paper, I 
use the term performance measures in hbraries as 
meaning quantified data on library activities . 

3.2 PURPOSE 

Quite simply, the purpose of performance measure- 
ment at the CBSL is: to provide data for the assess- 
ment of 1) whether or not the library fulfils its goals 
and objectives, and 2) the continuing relevance of 
these goals. 

3.3 TYPES 

There are various typologies of performance mea- 
sures. 

Input / process / output measures : Input measures 
relate to the resources of the hbrary in terms of 
money, staff, collections etc. Process data concern 
the internal functions of the library (eg. processing 
time for new books), and output data describe its 
service product (eg. number of books lent). 

Traditional library statistics have concentrated on 



basic input and output measures. The modem devel- 
opments in performance measurement have includ- 
ed process data and extended the area of output 
data. As efficient internal process functions are a 
basic condition for good service, they should not be 
neglected in a comprehensive program of library 
performance measurement. 

Objective / subjective measures : the subject- 
object problem is a basic problem of philosophy and 
psychology, which I happily leave to the speci^sts, 
cf. my comments above. 

For our purpose, I consider objective measures as 
relating to physical things (eg. number of books) 
and subjective measures as relating to psychological 
states (eg. user satisfaction). 

I do not consider objective measures to be more 
true than subjective measures. Both are necessary, 
but even together and even when obtained under 
optimal conditions they will not give a complete 
and ‘true’ picture. Angels would not be happy, but 
library managers wiU be pleased to obtain sensible, 
however imperfect data with which to plot their 
course in the murky-muddy waters of everyday 
operations. 

An interesting development at our institution (the 
Business School) has been an experiment with 
obtaining subjective measures of the same items 
from various stakeholder groups. At the CBSL it 
has been tried out by submitting questionnaires on 
various aspects of service not only to users but also 
to the library staff. Interesting differences of opinion 
between users and staff have been demonstrated. In 
one case, users were happy with the service at the 
information desk, but staff felt there were serious 
problems eg. relating to new media (CD-ROM) and 
they were definitely less happy with their service to 
users than users seemingly were. Another case was 
the OPAC, where users expressed satisfaction with 
the user-friendliness of the system, while staff were 
not very impressed by users’ OPAC-competence. 
This difference prompted a special investigation of 
a qualitative nature which concluded that users - 
contrary to users’ own perceptions - are not very 
good at using the system and that they are, for this 
reason, far from profiting optimally fi'om the infor- 
mation resources which the hbrary puts at their dis- 
posal. 

Non-comnarative and comparative : perfonnance 
measures concerning one institution in a single year 
are of course interesting in themselves. However, 
they become much more interesting when they can 
be compared with data from other years and from 
other institutions. 

Comparative data are fraught with their own kind 
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of problems. Due to changes or differences in defin- 
ition and practice, comparison across time and insti- 
tutions may be difficult, sometimes extremely so. 
Still, they should be tried, and problems solved as 
they arise or are recognised. 

Whether comparisons between institutions’ per- 
formance measures can be considered to be ‘bench- 
maridng’ is a matter for debate. If one includes 
qualitative analysis of best practice in the definition 
of benchmaridng, mere comparison of quantitative 
data may not suffice. Such data must be seen in 
their relationship to libraries’ mission, goals and 
objectives and they do not provide answers to why 
and how one library has ‘better’ scores than another. 
Still, it is interesting to create a horizon of compari- 
son and the comparison of quantitative data may 
also be quite relevant as first steps in a process of 
identifying, describing and analysing ‘best practice’. 

Simple and composite : simple measures are 
direct measures of something. Composite measures 
are obtained by relating simple measures, usually by 
division. 

Composite measures (sometimes termed as indi- 
cators in contradistinction to measures) are neces- 
sary for a deeper analysis of phenomena and they 
are essential to comparison over years and across 
institutions. It is interesting to know that a library 
lends 300,000 books a year. It is even more interest- 
ing to know that it lends ten books per FTE student, 
especially if you want to know how the library per- 
forms in relation to other libraries. To know that one 
library lends 400,000 books and another 100,000 is 
not very interesting in itself. What you would really 
like to know is how the circulation figure relates to 
the budgets of the libraries and the number of stu- 
dents - and the rate of registered use to imregistered 
use in reading rooms. And then you would stiU need 
to know the basic acquisition policies of the institu- 
tions, of course: do they acquire books to cover the 
urgent needs of students mainly, or do they acquire 
books ‘for the centuries and the scholar showing up 
in 200 years’, or do they acquire books because they 
are responsible for preserving the national literary 
culture? 

4. A Case: Interlibrary Loans from 
Other Libraries 

4.1 INTRODUCTION 

In order to illustrate the points and ideas developed 
above, I shall now present a case. It is based on a 
library resembling the CBSL, but the data given 
below are not identical with ours, and they are 



selected with a view to clarify certain basic notions. 

Imagine a library being part of a higher educa- 
tional institution and serving a primary user group 
of 10,000 students (FTEs) and 600 academic staff. 
Its yearly budget is 2.5 million poimds and it spends 

350.000 poimds on acquisitions, including 10,000 
pounds on ILL from other libraries. Its collections 
amount to 250,000 volumes. It pursues an active 
weeding policy and aims at a no-growth library of 

300.000 volumes in the year 2(X)0. 

The activity concerning interlibrary loans from 
other libraries has been stagnating for a number of 
years. Formerly, almost all loans were for teachers 
at the CBS; in the most recent years there has been 
an important relative rise in the number of loans for 
students. It is believed that the last development is 
due to students becoming familiar with the large 
bibliographic databases on CD-ROM and in the 
diminution of the - already - small fee for IL loans 
from abroad. This fee is not imposed in order to 
recover costs but to regulate behaviour and to 
ensure that only items really needed are requested. 

Whereas teachers often do not need very speedy 
delivery of materials borrowed from other libraries, 
the students need rapid delivery because the materi- 
als are to be used in papers to be finished within 
weeks or a few months - or yesterday! 

The library is stiU predominantly using traditional 
methods of ILL. Only recently has it begun to use 
online and fax-based methods of delivery and sys- 
tems like UNCOVER are stiU only used to a very 
limited extent. 

4.2 OBJECTIVES 

The objective of our fictive Ubrary concerning inter- 
library borrowings is the foUowing: 

- To provide reUable and speedy loans from other 
libraries and information systems 

The success criterion is: 

- A fiU rate of 95% of aU requisitions received 
from users and a mean delivery period of 21 
days 

4.3 THE PERFORMANCE MEASURES 

The performance measures are given in three tables, 
one with objective data, one with subjective data 
and one with comparative data. 

In Table 1 the ‘fiU rate’ is the number of requisi- 
tions from users fulfiUed within standard delays. 

‘IL loans/aU loans %’ means loans to users of 
materials borrowed from other libraries as a per- 
centage of aU loans. 

It is seen in this table that: 
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Table 1 



Objective performance 


measures of IL borrowings 
1992 1993 1994 


For teachers 


2,500 


2,500 


2,500 


For students 


500 


600 


700 


All 


3,000 


3,100 


3,200 


FiU Rate 


93% 


94% 


95% 


Delivery Time (mean) / days 


25 


23 


21 


IL borrowings / all loans 


1.0% 


0.9% 


0.8% 


Cost per loan / D.Kr. 


75 


77 


79 



- the number of interlibrary borrowings is increas- 
ing very slowly, and only due to an increase of 
borrowings for students 

- The fill rate has risen over the last years to just 
fulfil the strategic requirement of 95% 

- The delivery time has fallen over the last years to 
just fulfil the strategic requirement of 21 days 

- The interlibrary borrowings activity forms a very 
small and decreasing part of the total lending 
activity of the library 

- The cost of interlibrary borrowings is going up a 
little, but is stagnating if inflation is accounted 
for 

Table 2 gives the (very!) fictive satisfaction rates of 
4 groups: 2 user groups, ie. academic staff and stu- 
dents, the library staff, and the library management. 
The indifferent, unsatisfied and very unsatisfied 
users are not given - for simplicity’s sake. 

It is seen in this table that concerning the fill rate 

- teachers are consistently and highly (99%) satis- 
fied 

- students are consistently satisfied at the 80% 
level 

- library staff satisfaction increased over the years 
from 50% to total satisfaction coinciding with 
the achievement of the strategic objective 

- library management only became satisfied when 
the strategic objective was fulfilled in 1994 

Concerning the times of delivery 

- teachers were consistently satisfied at the 90% 
level 

- students’ satisfaction level decreased from 80% 
to 40% 

- library staff satisfaction followed the same pat- 
tern as for the fill rate 

- library management satisfaction fell from 50% to 




Table 2 



Subjective performance measures of IL borrowings 


Satisfied or very satisHed 
Teachers 


1992 


1993 


1994 


Fill Rate 


99% 


99% 


99% 


Delivery Times 
Students 


90% 


90% 


90% 


Fill Rate 


80% 


80% 


80% 


Delivery Times 


80% 


60% 


40% 


Library Staff 








Fill Rate 


50% 


75% 


100% 


Delivery Times 
Library Management 


50% 


75% 


100% 


Fill Rate 


0% 


0% 


100% 


Delivery Times 


50% 


25% 


0% 



Table 3 



Comparative performance 


measures of IL borrowings 


Lib A 


LibB 


LibC 


LibD 


All IL borrowings 3,200 


4,500 


5,000 


5,500 


IL borrowings / all loans 0.8% 


5.0% 


2.0% 


4.0% 


Cost per loan / D.Kr. 79 


120 


95 


102 



In Table 3, library A is our case library, and the oth- 
ers are libraries of the same type and those which 
are most often used by library A for comparison. 

It is seen that 

- the case library has the smallest number of inter- 
library borrowings both in absolute numbers and 
as part of the total lending activity 

- it also has the smallest cost per borrowing 

4.4 EVALUATION ON THE BASIS OF 
PERFORMANCE MEASURES 

The first question addressed in the evaluation exer- 
cise is the following: 

Is the actual practice of the library in conformity 
with the strategic objective and the success criteri- 
onZ 

The answer is: yes. It is satisfactory that the library 
has now come to fulfil its strategic objective con- 
cerning interlibrary borrowings. 

The second question is: 

Do all the available data tjoint to a satisfactory situ- 
ation concerning interlibrarv borrowings? 



10 



58 



Michael Cotta-Schenberg: PM in the Context of Quality Management 



The answer is: no. The decreasing satisfaction of 
students and management with the time of delivery 
is not satisfactory, even if the strategic objective is 
fulfilled. 

Fictive foUow-up studies of this phenomenon show 
that students are simply getting more demanding, 
particularly as they get more attuned to instant 
online contact with an increasing number of infor- 
mation resources. Management is unhappy because 
major systems for rapid (24-48 hours) online or fax 
delivery of periodical articles have now become 
available with the result that delivery times which 
were satisfactory in the earlier period have now 
become unsatisfactory. 

The third question is: 

How would you evaluate the activity concerning 
interlibrarv loans on the basis of these data? 

It is satisfactory that the strategic objectives are now 
being met. However, the success criterion concern- 
ing the delivery time has been made obsolete by the 
information technological development and should 
be revised. 

The fourth question is: 

If you have mentioned any problems in the evalua- 
tion. how would you propose to address them? 

Three types of corrective action have been men- 
tioned earlier: improving the data, revising strategic 
objectives or their success criteria, undertaking spe- 
cific initiatives of development. 

In our case, all three will be employed. 

- A special investigation of how many requisitions 
actually concern periodicals available on high- 
speed document delivery systems will be made. 
If a sufficient number proves to be available, 
then 

- the success criterion for speedy interlibrary bor- 
rowings will be lowered from 21 to 14 days (as 
a cautious beginning) 

- ILL procedures will be changed so that 
UNCOVER and similar systems will be used in 
preference and if possible to other, traditional 
partners 

The main notion to be illustrated by this case is that 
performance measures should be used both for 
assessing whether strategic objectives are being met 
and if strategic objectives remain relevant. Often, 
subjective performance measures may be the best 
indicator of the continued relevance of strategic 
objectives. 



5. Problems of Performance 

Measurement Affecting Quality 
Management 

O ne thing with performance measures is 
unavoidable: you can never feel safe! Some 
definition suddenly turns out to cover aspects you 
had not thought about, somebody ‘on the floor’ 
changes the way of counting something, the demon 
in your machine develops acute insanity - as when a 
computer at a major library started to count only 
every second loan - or there has been a misunder- 
standing between managers and programmers con- 
cerning some parameter of automatically produced 
management information. 

Before going into various problem types, a piece 
of advice: stay calm! Try to control the information 
gathering process, start with clear definitions, estab- 
lish simple procedures, have someone in charge 
who has a good general knowledge of your library’s 
functions and a feeling for numbers and who can 
sense when there must be something wrong with the 
data. This advice is so good that I shall eventually 
follow it myself! 

However, whatever you do the error-free system 
is impossible. Recognise that and simply correct 
errors and misunderstandings as they are discov- 
ered. Sometimes, you may have to declare certain 
data lost and you will lose comparability over the 
years. WeU, life goes on. 

Also, if you are experimenting with new forms of 
complex data you may reach a point where it 
becomes necessary to declare that the problems in 
defining and collecting them cannot be solved ade- 
quately, something still stinks. In that case, cut the 
measurement. Even if it may be embarrassing in 
view of your earlier efforts, possibly in the face of 
staff opposition. 

5.1 VALIDITY 

By validity is meant simply that the measure actual- 
ly measures what it is supposed to measure. 

An example: We have been measuring the time of 
delivery of books from our book-agents, counted 
from the day our order is sent to the day the book is 
received. The time consistently turned out to be 
(and stiU is) surprisingly high. However, only after a 
couple of years we became aware that the CBSL 
orders quite a large number of books, on the basis of 
pre-announcements, which have not been published 
at the time of ordering. Our measurement of deliv- 
ery time had thus somehow been messed up with 
delays of publication. The problem was solved by 
excluding books, which we knew had not been pub- 
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lished at the time of ordering, from the measure- 
ment. We could not solve the problem of books 
which had actually not been published but we 
thought they had. StiU the measurement today is 
more valid than it was before we made the correc- 
tion for books known to be unpublished. 

Another example: the fill rate for requisitions of 
books through interlibrary loan fi’om other libraries 
is calculated on the basis of number of requisitions 
sent to other libraries (B) and the number of books 
etc. received (A) - as A/B%. Three problems have 
been discovered. Firstly, it turned out out that if we 
did not get the material requested from a library, we 
would send the requisition to another library. We 
would then count the requisition twice. As we only 
obtained the material once, the fill rate would be 
50%. However, there would only be one request 
from the user, and she actually obtained it! So we 
had a problem of validity. We solved it by starting 
to count only the first requisition to a library, and 
not the repeat-requisitions to other libraries. 

Secondly, at some point it turned out that users’ 
requests were also counted as ILL requests in those 
cases when our library decided to buy the book 
requested. As the request was counted as an ILL 
request by the ILL department, but the book was not 
received by that department (but eventually by the 
acquisitions department), the book was not counted 
as a fulfilled ILL request. So the fill rate for this 
request was 0%, even though the book would actu- 
ally be obtained for the user. Again a problem of 
validity. We solved it by starting not to count 
requests transferred from ELL to acquisitions. StiU, 
how should we deal with user-requests transferred 
to the acquisitions department when the book was 
not actuaUy received from the bookseUer? At pre- 
sent we have chosen to ignore the problem, so we 
StiU have a validity problem somewhere, but the 
pure ILL measure is more valid now. 

Thirdly, a number of ILL requests are based on 
faulty and unverifiable bibliographic information 
from the users. They concern unexisting materials 
or materials which exist by another name. As the 
Ubrary’s requisitions cannot be met, these user 
requests lower the fiU rate to an unknown degree. 
However, as we now have a fiU rate of 98% of our 
ILL requisitions to other Ubraries, we have decided 
to ignore the problem. 

So, you wiU not be suiprised that having discov- 
ered three problems with this fairly simple measure 
I cannot help wondering what problems we might 
not yet have discovered in this regard. 

12 



5.2 RELIABILITY 

Is a measurement performed consistently and cor- 
rectly? 

An example: One of the departments of the 
CBSL was responsible for reporting monthly the 
number of article copies sent to other libraries. The 
monthly number was usuaUy about 20. Then, one 
month the number rose to about 200. 1 noticed it but 
did not have it checked, as it might be because of 
some special order fi'om another institution. When 
the c. 200 figure was repeated during the 
next two months I asked the head of the department 
what was happening. She answered that other 
libraries were discovering the excellent holdings of 
the department and the promptitude with which 
requests were fulfilled. Not an eye was dry, as we 
say in Danish. However, a week after she called 
back, very embarrassed, that actually it was a new 
technical assistant who had started to count the 
number of photocopied pages instead of articles. 

These things will happen all the time, and some 
of the problems will reflect the way your library 
trains new staff members, such training being of 
course an essential element of quality management. 

Sometimes the unreliable (or invalid) measures 
give a false-better image of your library’s doings, 
sometimes a false-worse one. At any rate, the 
greater the unreliability of your measurements, the 
less you are able to ^sess the functions of your 
library and whether you are meeting your ovm goals 
and objectives. You can then only hope that you do 
not run into a major uncharted iceberg-problem and 
sink with aU hands. 

5.3 VERBALISATION 

For the benefit of your audience - the readers of 
your quality report, staff, university management 
etc. - you want to verbalise those complicated fig- 
ures in the tables. 

‘This table shows that . . . ’ 

The first problem arising is one of translating fig- 
ures into words. When is a 25% increase a ‘large 
increase’, when is it ‘small’? When is it ‘significant’ 
or ‘considerable’? Even if you do take care about 
using such words consistently and in consideration 
of the underlying absolute figures or their degree of 
complexity (eg. the use rate of your collection), you 
may easily slip into using language which somehow 
reflects your personal values. As the measures 
should provide a solid basis for a common under- 
standing of the situation of the library, this is a prob- 
lem. 
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5.4 INTERPRETATION 

A most common error concerns causality. A certain 
development is clearly indicated by some figures, 
and you triumphantly or dejectedly conclude that 
this or that change of policy or unfortunate external 
event was responsible for the development. 

An example: recently the CBSL decided to lower 
the fee for articles printed from our fuU text busi- 
ness periodicals CD-ROM system. The change was 
followed by a ‘significant’ increase in the number 
of copies requested by users. As I had myself pro- 
posed the lowering of the fee, I easily concluded 
that the rise in the number of copies was due to the 
lowering of the price. My deputy, however, who is 
responsible for the library’s economy and who had 
opposed the measure, countered that the develop- 
ment was due to the quite predictable increase of 
students’ familiarity with the system. We may both 
be right, of course. 

The point is that the measure in itself did not teU 
us about the reason for the development of the fig- 
ures. 

It would be inhuman to forbid interpretation and 
explanations of performance measures, even con- 
cerning causality. However, care must be shown, 
and before you publicise your findings you might 
profit fiom having your interpretations examined by 
informed, intelligent people - preferably some who 
dislike you vehemently. You may suffer in the 
process, but you should end up with more refined 
interpretations. 

5.5 RECEPTION 

In the total quality process it is important how your 
performance measurement is ‘received’ by your 
stakeholders. 

In relation to the CBS , ie. our academic mother 
institution, it has been rather encouraging that the 
School authorities react positively towards the 
library’s efforts in the line of performance measure- 
ment and quality management. The credibility of 
the library as a service institution is raised, though 
sometimes - embarrassingly - our good intentions 
are confused with the actual thing. 

A problem is that external parties may easily fix- 
ate on certain very understandable measures, like 
the number of loans. As long as it goes up, fine. 
What will happen when it goes down because we 
shall be actively promoting our users’ direct con- 
nection with international online information 
providers, thus effectively teaching them to bypass 
the library? 

Another problem is unwanted comparisons with 
other, similar institutions. Why does this library 



have fewer loans per student than the library at X 
University? WeU, possibly because we have an 
abnormally high percentage of part-time students, 
for example. Actually, I have not met this type of 
reaction at our own institution, but I am told that it 
can be a real problem in countries, like England, 
where generalised systems for comparison of per- 
formance measures may be important for allocation 
of funds. 

At any rate, I do believe that a consistent and 
credible program of quality management and perfor- 
mance measurement wiU raise the library’s standing 
in the institution of which it forms part. 

In relation to library staff the situation is highly 
complex. 

Performance management may be resented by 
staff as basically a mechanism of control and a 
time-consuming waste of resources. The problem is 
aggravated in cases when the performance measures 
directly concern deficiencies and faults at staff 
level, eg. the number of cataloguing errors. 

In this respect libraries are very different, and 
general advice difficult. 

The much desired total mobilisation of staff 
required by the formal TQM systems I consider to 
be impossible in cultures with low degrees of 
authoritarianism and absence of the mystique of the 
leader. I do not consider it to be a coincidence that 
this requirement (of total staff mobilisation) was 
formulated in Japan and in the US. 

Personally, I distrust management by mystique, 
by seduction or by charismatic infusion of ‘motiva- 
tion’ by leaders into staff. 

Taught by experience, I am also sceptical about 
the possibility or desire of each individual staff 
member to relate to the total quality picture and 
management of the institution. 

I do believe that managers should make a reason- 
able and intelligent effort at having staff understand 
and participate in performance measurement and 
quality management. 

On the other hand, I feel that the claims made by 
the QM gurus concerning staff engagement in the 
total process are exaggerated. Many staff members 
can perfectly weU want to do a good job and actual- 
ly do it without subscribing to or being deeply inter- 
ested in managers’ grand plans. As long as they do a 
good job, they do not have to say, ‘Hail quality’ 
every morning. 

And the users ? Users are only dimly, if at aU, 
aware of quality programs and performance mea- 
surement What they react to is basically: can they 
get what they need - when they need it? Blissfully 
they ignore our wonderful intentions and are ordy 
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concerned with one thing: the actual service prod- 
uct. 

However, we have discovered that students do 
tend to react positively to user studies, particularly 
those which are carried out in a ‘personal’ atmos- 
phere - being questioned over a cup of coffee and a 
biscuit, preferably by a yoimg nice person of the 
opposite sex! Comments reflect the feeling that ‘it is 
good that the library cares’. 

There may be some problems of methodology in 
this and the effect - if it is true - is an unintended 
side-effect. However, if the library really does care, 
it is OK that the students get the message. 

6. Conclusion 

P erformance measurement is a necessary instru- 
ment for quality management. It makes possible 
the assessment of whether or not a library Mfils its 
goals and objectives, and whether or not particular 
objectives remain relevant. 

Performance measurement cannot provide a total- 
ly objective, ‘true’ and comprehensive image of a 
library’s function. A quality conscious library can- 
not do without it, though, even if it will always be 
flawed or incomplete. 

So, management will go on being an art, not a 
science, and decisions will still have to be made on 
the basis of approximate knowledge and intuition, 
not as results of computer calculations and 
imtouched by human minds. Fortunately! 



Discussion 

Don Revill, Liverpool John Moores University: On 
the subject of time and its importance: I do not 
believe time is of the essence. Time is something 
librarians think they ought to minimise. No other 
profession gives instant answers. I would say to a 
student that if it takes 21 days to get an interlibrary 
loan, they ought to have requested it earlier. It is 
said that taking time to deliver imbues it with 
importance. It was worth waiting for. Research has 
shown that time is not important to academics but it 
is to the students. They shouldn’t need interlibrary 
loans. Why worry if it takes a long time. It is some- 
body else’s performance that is at fault: the publish- 
ers, the transport system, the postal service. 

Michael Cotta-Sch0nberg: I respectfully disagree. 
There is always a fast way to do things. And stu- 
dents are important people too. The students’ time is 
important to them, so it is important to us. But to 



take one instance that I am keenly concerned about: 
if a student wants a book that we have actually 
bought, and if he or she has to wait six months 
before getting it because there is a queue for it, is 
that not important to the student? Should we not be 
concerned about it? 

Don Revill: That is a different problem. 

Michael Cotta-Sch0nberg: OK, but it is a time prob- 
lem. 

Roswitha Poll, Universitdts- und Landesbibliothek 
Munster: We are working on a national system of 
interlibrary lending now which will be very quick, 
‘Con Subito’. In a questiormaire to find out desired 
delivery times, we asked whether 24 or 48 hours 
was too quick for document delivery and asked 
users about payment for a quality service, say the 
equivalent of £1, £2 or £3 per document. We foimd 
that 50% to 80% were satisfied with two weeks 
delivery. 20% to 30% wanted material very urgently 
and would pay for it. Security - being sure of getting 
the document - seemed to be more important than 
getting it quickly. 
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